Automotive display validation

ABSTRACT

A method includes receiving, by a system-on-a-chip (SoC) from a camera mounted on a vehicle, a first image and transmitting, by the SoC to a display circuit over an interface cable, the first image. The method also includes receiving, by the SoC from the display circuit, a feedback signature corresponding to the first image. Additionally, the method includes detecting, by the SoC, an error, in response to determining that the feedback signature does not match the transmission-side signature and transmitting, by the SoC to the display circuit, a second image, in response to determining that the feedback signature matches the transmission-side signature.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional PatentApplication No. 62/554,105, filed on Sep. 5, 2017, entitled “A Methodand System to Detect End-to-End Frame Freeze and bad Pixel Data for aSafety Critical Automotive Display and Processor System,” whichapplication is hereby incorporated herein.

TECHNICAL FIELD

The present disclosure relates to a system and method for automotivedisplay, and in particular, to a system and method for automotivedisplay validation.

BACKGROUND

Many applications involve safety critical displays. For example, thetell tales of digital clusters are safety critical. Also, driven byadvanced safety features and driver assistance applications, theautomotive industry is increasing the amount of information shared withautomobile drivers. Examples of driver assistance applications includerear camera applications, surround view camera applications, and sideand rear view mirror replacement applications. Driver assistanceapplications involve displaying safety critical video footage capturedby a camera and other information to a display, to be viewed by a driverof the automobile. The display may include additional information addedto the video, for example parking lines in a parking application.

SUMMARY

An example method includes receiving, by a system-on-a-chip (SoC) from acamera mounted on a vehicle, a first image and transmitting, by the SoCto a display circuit over an interface cable, the first image. Themethod also includes receiving, by the SoC from the display circuit, afeedback signature corresponding to the first image. Additionally, themethod includes detecting, by the SoC, an error, in response todetermining that the feedback signature does not match thetransmission-side signature and transmitting, by the SoC to the displaycircuit, a second image, in response to determining that the feedbacksignature matches the transmission-side signature.

An example automotive display system includes a display circuit. Thedisplay circuit includes an interface configured to receive an imagefrom an SoC over an interface cable and a frame random access memory(RAM) coupled to the interface, the frame RAM configured to store theimage. The display circuit also includes light emitting diode (LED)drivers coupled to the frame RAM, the LED drivers configured to displaythe image on a display and a signature generator coupled to the frameRAM. The signature generator is configured to generate a feedbacksignature based on the image and transmit, to the SoC, the feedbacksignature.

An example circuit includes a histogram generation circuit. Thehistogram generation circuit includes a first configuration register binthresholds for B bins, where B is an integer greater than 1 and a secondconfiguration register indicating a portion of an image. The histogramgeneration circuit also includes an M-way comparator coupled to thefirst configuration register and to the second configuration register,the M-way comparator configured to generate M histograms of the portionof the image, the M histograms containing N bins, where M is an integergreater than 1, and a histogram merging circuit configured to generatean output histogram based on the M histograms, the output histogramcontaining N bins.

BRIEF DESCRIPTION OF DRAWINGS

For a more complete understanding of the present disclosure, and theadvantages thereof, reference is now made to the following descriptionstaken in conjunction with the accompanying drawings, in which:

FIG. 1 illustrates an example automotive display system;

FIG. 2 illustrates a flowchart of an example method of automotivedisplay validation performed by a system-on-a-chip (SoC);

FIG. 3 illustrates a flowchart for an example method of automotivedisplay validation performed by a display circuit;

FIG. 4 illustrates an example automotive display validation system;

FIG. 5 illustrates another example automotive display validation system;

FIG. 6 illustrates an additional example automotive display validationsystem;

FIG. 7 illustrates another example automotive display validation system;

FIG. 8 illustrates an additional example automotive display validationsystem;

FIG. 9 illustrates an example histogram generation circuit; and

FIG. 10 illustrates an example histogram comparison circuit.

DETAILED DESCRIPTION

Displaying information to drivers in automotive applications assists thedriver in making decisions based on high quality information. However,such a display imposes strict safety requirements on the displayedcontent. The safety requirements ensure the safety goals of the correctsafety critical data being displayed, the frame not being partially orfully frozen, and the image frame meeting the safety critical latencyrequirements. It is desirable for an automotive display system toguarantee accuracy, for example using automotive display validation.

FIG. 1 illustrates automotive display system 100, which a camera 102,system-on-a-chip (SoC) 104, display circuit 114, and display 126. In anexample, the camera 102 is mounted on a vehicle, such as an automobileor truck. The camera 102 may be a charge coupled device (CCD) videocamera or a complementary metal oxide semiconductor (CMOS) video camera,and may face in any direction. In one embodiment, a vehicle containsmultiple cameras facing in multiple directions.

The SoC 104 receives video data from the camera 102. In an embodiment,the SoC 104 is a TDAx™ driver assistance chip, produced by TexasInstruments. The SoC 104, which may be located close to the camera 102,contains processors, such as an image signal processor (ISP) 103, acentral processing unit (CPU) 106, and a digital signal processor (DSP)108. Three processors are pictured, but the SoC 104 may include multipleCPUs, multiple DSPs, or other processors, such as one or more embeddedvision engine (EVE) processors. The ISP 103 processes the raw datareceived from the camera 102. The CPU 106 and the DSP 108 performprocessing on the video data received from the camera 102. The SoC 104also includes a display subsystem (DSS) 110, which outputs the videodata processed by the CPU 106 and the DSP 108 to the display circuit114. In an embodiment, the video output data of the DSS is validatedduring processing.

The SoC 104 is coupled to the display circuit 114 by an interface cable112. The interface cable 112 may be an Ethernet cable, a low voltagedifferential signaling (LVDS) cable, a high definition multimediainterface (HDMI) cable, or another cable type. The interface cable 112may be many meters long. For example, one end of the interface cable 112may be in the rear of a truck, and the other end inside the cab of thetruck.

The display circuit 114 includes an interface 116, a frame random accessmemory (RAM) 118, a display controller 120, segment light emitting diode(LED) drivers 122, and a common LED drivers 124. In one embodiment, thedisplay circuit 114 is a printed circuit board (PCB). The interface 116receives a video signal transmitted from the SoC 104 over the interfacecable 112. The frame RAM 118 stores the video frames for display to auser, such as a driver. Also, the display controller 120 controls thedisplay by signaling the segment LED drivers 122 and the common LEDdrivers 124. The segment LED drivers 122 and the common LED drivers 124display the video stream to the user via the display 126, which may bean LED display. It is desirable to validate the display at locations inthe transmission path that occur after all or part of the interfacecable 112, at the display circuit 114, or at the display 126.

An example automotive display validation system detects errors in anautomotive display, such as frozen frames and bad pixel data, for safetycritical automotive applications. Some embodiments utilize exact methodsof automotive display validation that validate the entire display. Otherembodiments utilize approximate statistical methods of automotivedisplay validation that consider a portion of the display. In anembodiment, a hardware circuit is used to generate histogramson-the-fly.

FIG. 2 illustrates a flowchart 300 for an example method of automotivedisplay validation performed by a transmission-side SoC. In block 314,the SoC receives a video image from a camera. The camera may be mounted,pointing in any direction, on a vehicle, such as an automobile. In anembodiment, the SoC is located physically close to the camera.

In block 316, the SoC performs processing and analysis on the videoimage, to generate a processed image. A processor, such as an ISP,performs image processing on the video image received from the camera inthe block 314. For example, the processor performs Bayer transformation,demosiacing, noise reduction, and/or image sharpening. In Bayertransformation, the processor determines an RGB value for each pixelbased on a pattern designated by the Bayer filter. In demosiacing, theprocessor evaluates the color and brightness data of a pixel, comparesthe color and brightness data with the color and brightness data fromneighboring pixels, and uses a demosiacing algorithm to generate anappropriate color and brightness value for the pixel. The processor mayalso access the picture as a whole to ensure the correct distributionand contrast, for example by adjusting the gamma value. In noisereduction, the processor separates noise from the image to remove noise,for example by filtering the image. In image sharpening, edges andcontours are sharpened using edge detection. Image sharpening may beperformed to compensate image softening that was introduced by the noisereduction. In some embodiments, the block 316 is not performed.

In block 302, the SoC generates a transmission-side signature for theimage received in the block 314. The block 302 may be performed by aprocessor of the SoC, such as a DSP or a CPU. In some embodiments, thetransmission-side signature is a timestamp, a histogram, a cyclicredundancy check (CRC) value, a frame difference signature, or a securehash algorithm (SHA) value. In a frame difference signature, thesignature is the difference in value between two consecutive frames. Forexample, the processor may determine the absolute value of thedifference between the pixel values of two consecutive frames, and sumthese absolute value differences to determine the frame differencesignature. In another example, the processor determines the signed valueof the difference between the pixel values of two consecutive frames,and sums these signed difference values, to generate the framedifference signature. The processor may generate the transmission-sidesignature based on the video image received in the block 314 or based onthe processed image generated in the block 316. In some embodiments, theprocessor may generate the transmission-side signature based on anentire image or based on a portion of the image. In additionalembodiments, the processor generates the transmission-side signature byaltering the image. In one embodiment, the processor adds a referenceframe, such as an all-black frame or an all-white frame, to the videotransmission stream. In other embodiments, the processor alters aportion of the frame in a predetermined manner. For example, a patch ofwhite or black pixels may be added to a portion of the image not viewedby the user. In other embodiments, the processor adds an infraredpattern not visible to the user to the image. In additional embodiments,the processor selects a pseudo-random block from the image, and modifiesthe pseudo-random block. The processor may vary the location of thepseudo-random pattern used for selecting pseudo-random blocks from frameto frame to statistically cover the correctness of the whole displayregion. In some examples, the processor resizes the image beforegenerating the transmission-side signature. In some embodiments, theblock 302 is not performed. In other embodiments, the processor performsthe block 302 after transmitting the image, or after storing the image.

In block 304, the SoC transmits a transmitted image to a displaycircuit. The transmitted image may be the video image received in theblock 314 or the processed image generated in the block 316. Thetransmitted image may be transmitted by a DSS of the SoC, over aninterface cable, to an interface of the display circuit. In someexamples, the interface cable, which may be many meters long, is anEthernet cable, LVDS cable, or HDMI cable. In some embodiments, thetransmission-side signature is not transmitted. In other embodiments,the SoC separately transmits the transmission-side signaturetransmitted, or the SoC transmits the transmission-side signaturetogether with the image.

In block 305, the SoC stores a copy of the transmitted image in memory,such as in RAM. Additionally, or alternatively, the SoC stores thetransmission-side signature generated by the block 302 in the memory.

In block 306, the SoC receives a feedback signature, corresponding tothe image transmitted in the block 304, from the display circuit. Thefeedback signature indicates characteristics of the image which thedisplay circuit receives, processes, or displays. In one embodiment, theSoC receives the feedback signature over the same interface cable usedto transmit the transmission image to the display circuit. In anotherembodiment, a separate interface cable is used for receiving thefeedback signature. In an embodiment, the feedback signature is a CRCvalue, a frame comparison signature, or an SHA value. In someembodiments, the feedback signature is a timestamp. In otherembodiments, the feedback signature is an image. This image, which maybe smaller than the transmitted image, may be obtained by a camera whichis monitoring the display. In other embodiments, a demultiplexer, whichobtains the image, extracts all or part of the image before the image issent to the display controller. In some embodiments, the transmittedimage and/or the feedback image are resized to have the same scale. Insome embodiments, the feedback signature is a signal detected by a lightsensor over a portion of the image.

In block 308, the SoC determines whether the feedback signature matchesthe transmission-side signature. In one embodiment, the feedbacksignature matches the transmission-side signature when the feedbacksignature is the same as the transmission-side signature. In anotherembodiment, the feedback signature matches the transmission-sidesignature when the difference, or the sum of absolute difference,between the feedback signature and the transmission-side signature isless than a threshold value. In some embodiments, where the feedbacksignature is a feedback image, the SoC determines whether the feedbackimage matches the transmitted image. When the feedback signature matchesthe transmission-side signature, the SoC proceeds to block 310, anddetermines that there is no error. On the other hand, when the feedbacksignature does not match the transmitted image, there is an error, andthe SoC proceeds to block 312. In one embodiment, to determine whetherthe feedback signature matches the transmission-side signature, the SoCcompares the feedback signature to the transmission-side signature forthe transmitted or stored image. The SoC determines that thetransmission-side signature and the feedback signature match when theyare similar, but not identical, for example when the difference betweenthe transmission-side signature and the feedback signature is less thana predefined threshold. In another embodiment, the SoC only determinesthat the transmission-side signature and the feedback signature matchwhen they are identical. In an embodiment, to determine whether thefeedback signature matches the transmission-side feedback, the SoCperforms an analysis on the transmitted image and on a feedback image,and compares the results. In one embodiment, the feedback signature is afeedback image, and the SoC generates corner features for thetransmitted image and for the feedback image, and compares the cornerfeatures, to determine whether the feedback signature matches thetransmitted image. One or both of the images may be scaled beforegenerating the corner features, so the images have the same scale. Inanother embodiment, the SoC generates histograms for the transmittedimage and on the feedback image, and compares the histograms, todetermine whether the feedback signature matches the transmitted image.One or more histogram generation circuits may be used to generate thehistograms, and a histogram comparison circuit may be used to comparethe histograms. In another example, software generates and compares thehistograms. In one embodiment, the SoC compares a value from a lightsensor to an expected value based on a predetermined pattern, todetermine whether the feedback signature matches the transmitted image.

In block 310, the SoC determines that there is no error, and continuesto transmit images to the display circuit. For example, the SoCtransmits another image to the display circuit.

In block 312, the SoC determines that there is an error, and alterstransmission. In an embodiment, the SoC ceases to transmit images to thedisplay controller. Additionally, or alternatively, the SoC may transmitan error message to the user, indicating the nature of the error. Theerror message may indicate whether the error is a frozen frame or pixelerrors. In one embodiment, when the SoC detects an error, a seconddisplay, for example a lower resolution display using a secondary imagescreen, is used, instead of the primary image screen.

FIG. 3 illustrates a flowchart 320 for an example method of automotivedisplay validation, performed by a display circuit. In block 322, aninterface of the display circuit receives an image from atransmission-side SoC, for example over an interface cable.

In block 324, the display circuit stores the image received in the block322 in frame RAM.

In block 326, the display circuit displays the image stored in the frameRAM on a display visible to a user, for example a user who may bedriving a vehicle. A display controller may control segment LED driversand common LED drivers to display the image on the display. The displaymay be an LED display located near the user.

In block 328, the display circuit generates a feedback signature basedon the image. In one embodiment, the display circuit generates thefeedback signature by extracting the image from the frame RAM andgenerating the feedback signature based on the extracted image. Thefeedback signature may be a CRC value, a histogram, a timestamp embeddedin the image, a frame comparison between consecutive frames, or anothersignature, such as an SHA value. In an embodiment, the display circuitgenerates a timestamp as the feedback signature. In another embodiment,a video camera obtains the feedback signature and views the display. Inan additional embodiment, a light sensor located in a particular regionof the display generates the feedback signature. In another example, ademultiplexer generates the feedback signature, for example an LVDSdemultiplexer in the interface cable near the display circuit.

In block 330, the display circuit transmits the feedback signature tothe SoC. In one embodiment, the feedback signature is transmitted on thesame interface cable used to receive the image in the block 322. Inanother example, a separate interface cable is used.

FIG. 4 illustrates an automotive display validation system 130, whichimplements a method of automotive display validation based on the entireimage. In some embodiments, the automotive display validation system 130uses a low bandwidth feedback signature. The automotive displayvalidation system 130 includes video feedback 113 from within displaycircuit 140 to SoC 138. The SoC 138 receives an image, captured via thecamera 102 and the ISP 103. The CPU 106 and the DSP 108 process thereceived image. In one embodiment, the camera 102 is mounted on avehicle, such as an automobile. The SoC 138 transmits the image, usingDSS 110, to the display circuit 140, over interface cable 112. In oneembodiment, a clock on the SoC 138 generates a timestamp, which isassociated with the image. In one embodiment, the timestamp indicatesthe time that the SoC 138 transmits the image. In another embodiment,the timestamp indicates the time that the SoC 138 receives the image.The SoC 138 may also store a copy of the transmitted image in memory139, which may be a RAM. The ISP 103 performs image processing on theraw image received from the camera 102. For example, the ISP 103performs Bayer transformation, demosiacing, noise reduction, or imagesharpening. In Bayer transformation, the ISP 103 determines an RGB valuefor each pixel based on a pattern designated by the Bayer filter. Indemosiacing, the ISP 103 evaluates the color and brightness data of apixel, compares the color and brightness data with the color andbrightness data from neighboring pixels, and uses a demosiacingalgorithm to produce an appropriate color and brightness value for thepixel. The ISP 103 may also access the picture as a whole to ensure thecorrect distribution and contrast, for example by adjusting the gammavalue. In noise reduction, the ISP 103 separates noise from the image toremove noise, for example by filtering the image. In image sharpening,the ISP 103 sharpens edges and contours using edge detection. Imagesharpening may be performed to compensate image softening that wasintroduced by the noise reduction.

The display circuit 140 receives the image at the interface 116, fromthe SoC 138 over the interface cable 112. In one embodiment, the displaycircuit 140 adds a timestamp to the image at the time of arrival, usinga clock on the display circuit 140. The frame RAM 118 stores the image,which also proceeds to display controller 120. The image from the frameRAM 118 is shown by the display 126 using segment LED drivers 122 andcommon LED drivers 124.

Signature generator 136 extracts the image from the frame RAM 118, andgenerates a feedback signature. The feedback signature may be a CRCvalue, a comparison between frames, or another mechanism, such as an SHAvalue. In one embodiment, the signature generator 136 extracts atimestamp from the image. In another embodiment, the signature generator136 generates a timestamp. In one embodiment, the signature generator136 generates a histogram of the image. In an embodiment, the signaturegenerator 136 is implemented by a processor, for example a CPU. Thedisplay circuit 140 then transmits the feedback signature 113 to the SoC138. In one embodiment, the interface cable 112 transmits the feedbacksignature. In another example, a separate cable, for example a lowthroughput cable, transmits the feedback signature.

The SoC 138 reads the stored image from the memory 139. Also, in someembodiments, the signature generator 132 generates a transmission-sidesignature for the stored image. In an embodiment, the signaturegenerator 132 uses the same signature generation mechanism as thesignature generator 136. In some embodiments, the memory 139 stores thetransmission-side signature.

Signature comparator 134 compares the transmission-side signaturegenerated by the signature generator 132 of the transmitted image withthe feedback signature generated by the signature generator 136. In anembodiment, the signature comparator 134 compares the CRC value in thetransmission-side signature to the CRC value in the feedback signature,and a match is detected when the CRC values are the same. In anembodiment, the signature comparator 134 compares the timestamp of thetransmission-side signature to the timestamp of the feedback signature.The time lag for the image transmission and the asynchronous nature ofthe clock for the SoC 138 and the clock for the display circuit 140 maybe taken into account in comparing the timestamps. For example, thesignature comparator 134 subtracts a predetermined transmission timevalue from the feedback timestamp, to generate a compensated timestamp.Then, the signature comparator 134 determines the difference between thecompensated timestamp and the transmission-side timestamp. In anembodiment, the signature comparator 134 determines that the compensatedtimestamp matches the transmission-side timestamp when they are within apredetermined threshold of each other, to account for variations intransmission time. In one embodiment, the transmission-side signatureand the feedback signature must match exactly to determine a match. Inanother embodiment, the signature comparator 134 determines that thetransmission-side signature and the feedback signature match when theyare similar, and that they do not match when they are not similar, toaccount for noise. Relatively small amounts of noise in the image do notpose a safety threat. However, the signature comparator 134 detectssignificant mismatches, for example lost frames, which pose a safetythreat. The signature comparator 134 may send the results to the DSS 110to control further image transmission.

For example, the DSS 110 may halt image transmission when it detects anerror, and continues image transmission when it determines that there isno error. In one embodiment, additionally or alternatively, the DSS 110transmits an error message to the display circuit 114, indicating thenature or magnitude of the error. In one embodiment, the signaturegenerator 132 and the signature comparator 134 are implemented on aseparate processor. In other embodiments, the signature generator 132and the stamp and signature comparator 134 are implemented on the DSP108 or on the CPU 106. In another embodiment, the DSS 110 transmits anerror frame to the display and transmits tell-tales over a bus, such asa controller area network (CAN) bus, a multichannel audio serial port(McASP), a serial peripheral interface (SPI) bus, or the interface cable112. The tell-tales may be auditory or visual messages perceptible tothe user. For example, a warning light or alarm signal may alert theuser to a display error.

FIG. 5 illustrates an automotive display validation system 170, which isa method of automotive display validation based on considering theentire image. The automotive display validation system 170 includesvideo feedback 153 to SoC 172 from camera 152 trained on a display 126.In an embodiment, the automotive display validation system 170 isnonintrusive, and does not require modifications of the display circuit114. Additionally, the automotive display validation system 170 monitorsthe images actually being displayed on the display 126. An advantage ofthe automotive display validation system 170 is that the display circuit114 and the display 126 may be similar to display circuits and displaysthat do not have display validation.

The SoC 172 receives an image from the camera 102, which is mounted on avehicle, such as an automobile. Processors of the SoC 172, such as theISP 103, the CPU 106, and the DSP 108, process the image. The DSS 110transmits the processed image to the display circuit 114 over theinterface cable 112. Also, the memory 139, for example RAM, stores thetransmitted image. The interface cable 112 may be an Ethernet cable, anLVDS cable, or an HDMI cable.

In the display circuit 114, the interface 116 receives the imagetransmitted from the SoC 104 over the interface cable 112. The displaycontroller 120 controls the display 126 by signaling the segment LEDdrivers 122 and the common LED drivers 124. Also, the frame RAM 118stores the video frames for display. The segment LED drivers 122 and thecommon LED drivers 124 display the video to a user via the display 126,which may be an LED display. The camera 152 monitors the displayedimage. The camera 152 may be a CCD video camera or a CMOS video camera.In one embodiment, the camera 152 has a lower resolution than the camera102. The camera 152 feeds back the acquired images to the SoC 172 asimage feedback 153. In one embodiment, the image feedback is transmittedover the interface cable 112. In another example, the image feedback istransmitted over a separate cable.

The SoC 172 receives the images from the camera 152. In the SoC 172,video scaler 174 retrieves the image stored in the memory 139 and scalesthe stored image and/or the received image, so the two images have thesame scale. In some embodiments, the video scaler 174 scales thetransmitted image to generate a scaled image, and the memory 139 storesthe scaled image. In some embodiments, for example in some embodimentsthat use histograms, the video scaler 174 is not used.

A feature generator 176 generates features for both the received imageand the stored image. In one embodiment, the feature generator 176 andthe feature comparator 178 are implemented by a processor, such as theCPU 106, the DSP 108, or another processor. In another embodiment, forexample when histograms are used, the feature generator 176 and featurecomparator 178 are implemented in dedicated hardware, for example ahistogram generation circuit and a histogram comparison circuit. In oneembodiment, the feature generator 176 generates corner features, orpoints of interest, for the received image and for the transmittedimage. A corner may be defined as the intersection of two edges or apoint for which there are two dominant and different edge directions ina local neighborhood of the point. Corner detection may be performedusing correlation, Moravec corner detection, Harris Stephens cornerdetection, Forstner corner detection, the multi-scale Harris operator,or another approach, such as the level curve curvature approach,Laplacian of Gaussian approach, scale-space interest points, Wang andBrady corner detection, the smallest univalue segment assimilatingnucleus (SUSAN) corner detection, the Trajkovic and Hedley detection,the accelerated segment test (AST) based feature detection, or with theautomatic synthesis of detectors.

In another embodiment, the feature generator 176 generates histogramsfor the received image and for the transmitted image. The histogram maybe an intensity histogram for the image or color histograms. The featuregenerator 176 divides the value ranges into bins, and places each pixelin the corresponding bin. The bins may be uniform in size, or they mayvary in size.

The feature comparator 178 compares the features generated in thefeature generator 176 for the transmitted frame to the feedback image.When corner features are used, the feature comparator 178 compares thecorner features of the transmitted frame to the corner features of thefeedback image. Likewise, when histograms are used, the featurecomparator 178 compares the histogram of the transmitted frame to thehistogram of the feedback image. The feature comparator 178 maynormalize the histograms so they have the same scale. Each frame mayhave a unique histogram, so matching histograms indicates matchingframes. In some embodiments, the corner features or histograms mustmatch exactly to determine a match. In another example, sufficientlysimilar corner features or histograms indicate a match. For example, thefeature comparator 178 may determine that histograms match when the sumof absolute difference between the normalized histograms is less than apredetermined value. In another example, the feature comparator 178determines that corner features match when difference in magnitudeand/or location of the corner features is within a predeterminedthreshold. The feature comparator 178 sends the comparison results tothe DSS 110. The feature comparator 178 also sends the comparisonresults to the DSP 108 and the CPU 106. The DSP 108 and the CPU 106assist in detecting scenario errors, and improve robustness.

The DSS 110 may halt image transmission when it detects an error, andcontinue image transmission when it determines that there is no error.Additionally or alternatively, the DSS 110 transmits an error message tothe display circuit 114, indicating the nature or magnitude of themismatch.

FIG. 6 illustrates automotive display system 180, which uses an LVDSdemultiplexer (demux) 182 to extract a portion of the transmitted imagefrom an LVDS cable 188. The automotive display system 180 is a method ofautomotive display validation based on the entire image. Advantages ofthe automotive display system 180 include a low cost. Another advantageof the automotive display system 180 is that the display circuit 114 andthe display 126 may be similar to display circuits and displays that donot have display validation.

The camera 102, which is mounted on a vehicle, transmits an image to SoC184. The SoC 184 includes processors, such as the ISP 103, the CPU 106,and the DSP 108, that process the image and store the image in thememory 139. The DSS 110 transmits the image to the display circuit 114over the LVDS cable 188. The SoC 184 may also store the transmittedimage in the memory 139.

The LVDS demultiplexer 182 extracts an image frame from the LVDS cable188. In one example, the LVDS demultiplexer 182 may sample the imagedata during the extraction process. The LVDS demultiplexer 182 sends theextracted frame back to the SoC 184 as a feedback signature 183. In oneembodiment, the feedback signature is transmitted over the LVDS cable188. In another example, the feedback signature is transmitted over aseparate cable. In one embodiment, as pictured, the LVDS demultiplexer182 is before the display circuit 114. In another embodiment, the LVDSdemultiplexer 182 is at the input of the display circuit 114.

The interface 116 of the display circuit 114 receives the video signaltransmitted from the SoC 184 over the LVDS cable 188. The frame RAM 118stores the video frames for display to the user. Additionally, thedisplay controller 120 controls the display by signaling the segment LEDdrivers 122 and the common LED drivers 124, so the segment LED drivers122 and the common LED drivers 124 display the video to a user via thedisplay 126.

The SoC 184 receives the feedback signature from the LVDS demultiplexer182. In the SoC 184, a video scaler 186 scales the stored or transmittedimage and/or the feedback or received image, so the two images have thesame scale. In some embodiments, the video scaler 186 scales thetransmitted image to generate a scaled image, and the memory 139 storesthe scaled image. In some embodiments, for example when histograms areused, the video scaler 186 is not used.

The feature generator 158 generates features for both the received orfeedback image and the transmitted or stored image. In one embodiment,the feature generator 158 and the feature comparator 160 are implementedby a processor, such as the CPU 106, the DSP 108, or another processor.In another embodiment, for example when histograms are used, the featuregenerator 158 and the feature comparator 160 are implemented indedicated hardware as a histogram circuit. In one embodiment, thefeature generator 158 generates corner features, or points of interest,for the received image and for the transmitted image. In anotherembodiment, the feature generator 158 generates histograms for thereceived image and for the transmitted image.

The feature comparator 160 compares the features generated in thefeature generator 158 for the transmitted image and the feedback image.When corner features are used, the feature comparator 160 comparescorner features of the transmitted image to corner features of thefeedback image. Likewise, when histograms are used, the featurecomparator 160 compares the histograms of the transmitted image tohistograms of the feedback image. In some embodiments, the cornerfeatures or histograms must match exactly to determine a match. Inanother example, sufficiently similar corner features or histogramsindicate a match. For example, a sum of absolute difference may be takenbetween histograms, and a match is detected when the sum of absolutedifference is less than a pre-determined threshold. The featurecomparator 160 sends the comparison results to the DSS 110. The featurecomparator 160 also sends the comparison results to the DSP 108 and theCPU 106. The DSP 108 and the CPU 106 assist in detecting scenarioerrors, and improve robustness.

The DSS 110 may halt image transmission when it detects a fault, andcontinues image transmission when it does not detect a fault.Additionally or alternatively, the DSS 110 transmits an error message tothe display circuit 114, indicating the nature or magnitude of themismatch.

FIG. 7 illustrates an automotive display validation system 200 forautomotive display validation utilizing feedback from a light sensor202, where only a portion of the image is used for validation. In anembodiment, the automotive display validation system 200 is a low costsystem.

An SoC 204 receives an image from the camera 102, which may be mountedon a vehicle. Processors of the SoC 204, such as the ISP 103, the CPU106, and the DSP 108, process the image. The SoC 204 adds a referenceimage 206 to the image before transmission, to generate a modifiedimage. In one embodiment, the reference image is added only to a portionof the image that will not be visible to the user, to avoid disruptingthe experience of the user. The reference image may be a white patch, ablack patch, or a predefined pattern. In other embodiments, thereference image is an entire frame, for example an all-black frame or anall-white frame. The reference image may be an infrared image that isnot visible to the user. After the reference image has been added,memory 212 stores the modified image, and the DSS 110 transmits themodified image to the display circuit 114 over the interface cable 112.In some embodiments, additionally, or alternatively, the memory 212stores the reference image itself. In one embodiment, the original imagewithout the reference image and the reference image are storedseparately in the memory 212.

In the display circuit 114, the interface 116 receives the video framestransmitted from the SoC 204 over the interface cable 112. The frame RAM118 stores the video frames for display to the user. Also, the segmentLED drivers 122 and the common LED drivers 124 display the video via thedisplay 126, which may be a LED display.

A light sensor 202 senses the light in all of or a portion of thedisplay 126. In one embodiment, the light sensor 202 senses light in aportion of the image that is not visible to the driver or to the user.In one embodiment, the light sensor 202 is a simple photodetector, suchas a photodiode, phototransistor, or light dependent resistor (LDR). Inother embodiments, the light sensor 202 is a video camera, for example alow resolution video camera. In additional embodiments, the light sensoris a typical CCD or CMOS video camera. A low cost light sensor may beused. The light sensor 202 transmits the feedback signature 203, whichmay be based on the light measurement, to the SoC 204, either over theinterface cable 112 or over another cable.

The SoC 204 receives the feedback signature from the light sensor 202.The reference image extractor 210 extracts the reference image from thefeedback signature received from the light sensor 202. In oneembodiment, the reference image extractor 210 subtracts the originalvideo image from the memory 212 from a feedback image, to generate thecomparison image.

The SoC 204 also includes a reference image comparator 208, whichcompares the comparison image extracted in the reference image extractor210 to the reference image, to the original image, or the modified imagestored in the memory 212. The comparison image may be compared to thereference image. In an embodiment, a match is detected when a sum ofabsolute differences between the comparison image and the referenceimage is less than a pre-determined threshold. In some embodiments, thereference image comparator 208 only compares certain predefined frames,which contain reference images. When the extracted reference imagematches the generated reference image, the system determines that thereis no error. On the other hand, when the extracted reference image doesnot match the generated reference image, the system determines thatthere is an error. The reference image comparator 208 sends the imagecomparison to the CPU 106, the DSP 108, and/or the DSS 110.

The DSS 110 may halt image transmission when the system detects anerror, and continue image transmission when the system determines thatthere is no error. Additionally or alternatively, the DSS 110 transmitsan error message to the display circuit 114, indicating the nature ormagnitude of the mismatch.

FIG. 8 illustrates an automotive display validation system 220 for amethod of automotive display validation based on a portion of the image.The automotive display validation system 220 includes an SoC 226, whichtransmits an image, captured via the camera 102 and processed by the ISP103, the CPU 106, and the DSP 108, to the display controller 120, usingthe DSS 110 to transmit the image over the interface cable 112.

A pseudo-random address generator 230 pseudo-randomly, or semi-randomly,selects one or more regions within the image frame for modification. Inone embodiment, the pseudo-random address generator 230 selects apseudo-random block within the image frame by generating the location ofthe pseudo-random block. This pseudo-randomly selected block may varyfrom frame to frame, so that different regions are tested for differentframes. In one embodiment, the block locations are changed every framewith an explicit target to test sweep the full display within a giventime interval. The location of the pseudo-random block may be added tothe image frame, for example as metadata. In another example, thelocation of the pseudo-random block is maintained separately from theimage.

A pixel pattern generator 228 of the SoC 226 generates a predeterminedpixel pattern, or a signature, at the location of the image generated bythe pseudo-random address generator 230. In an embodiment, a limitednumber of pixels are adjusted in a predetermined pattern within theblock. The predetermined pattern may be designed to be not visible, orto minimally visible, to the user. For example, an infrared pattern maybe generated. The pixel pattern generator 228 adds the pre-determinedpixel pattern to a video image, to generate a modified image. In anembodiment, the pixel pattern generator 228 adds the location of thepseudo-randomly selected block to the modified image, for example asmetadata. The pre-determined pixel pattern adds some noise to the image,without significantly interfering with the user experience. Also, insome embodiments, a clock on the SoC 226 generates a timestamp and addsthe timestamp to the modified image, for example as metadata. Memory 139stores the modified image. In an embodiment, the memory 139 stores thepre-determined pixel pattern. Also, the memory 139 stores thepre-determined pixel pattern separately from the modified image. In oneembodiment, both the modified image and the unmodified image are storedin the memory 139. The timestamp for the image may also be stored in thememory 139. In an additional embodiment, the location of thepseudo-random block is stored in the memory 139. Additionally, the pixelpattern generator 228 sends the modified image to the DSS 110 fortransmission to a display circuit 222.

The display circuit 222 receives the modified image at the interface116, from the SoC 226, over the interface cable 112. The modified imagemay include embedded information indicating the location of thepseudo-random block for this frame, or other information indicated amodified portion of the image. In one embodiment, the display circuit222 adds a timestamp to the image at the time of arrival using a clockof the display circuit 222. The frame RAM 118 stores the modified image,and the modified image also proceeds to the display controller 120. Themodified image in the frame RAM 118 is shown by the display 126 usingthe segment LED drivers 122 and the common LED drivers 124.

A signature generator 224 extracts the image from the frame RAM 118. Thesignature generator 224 may extract a portion of the image indicated bythe location of the pseudo-random block. Then, the signature generator224 masks the image at the pseudo-random block. In an embodiment, thefeedback signature indicates the pixel values in the pseudo-randomblock. In another embodiment, the signature generator 224 assigns theentire image to be the feedback signature. In one embodiment, thesignature generator 224 also extracts a timestamp for the time oftransmission from the image, or assigns a timestamp to the image. In anembodiment, the signature generator 224 is implemented by a processor,for example a CPU. The display circuit 222 then transmits the feedbacksignature 225 to the SoC 138. In one embodiment, the interface cable 112is used for transmitting the signature. In another embodiment, aseparate cable, for example a low throughput cable, is used for thefeedback signature.

The SoC 226 reads the modified image from the memory 139. The SoC 226may also read out the pixel pattern and/or the pseudo-random blocklocation from the memory 139. In one example, the SoC 226 extracts thepixel pattern from the modified image using the pseudo-random blocklocation. In an embodiment, the SoC 225 reads out a timestamp from thememory 139. A signature comparator 232 receives the feedback signaturefrom the signature generator 224. The signature comparator 232 comparesthe modified image or the pixel pattern to the feedback signature. In anembodiment, the SoC 226 subtracts the original unmodified image from thefeedback signature, leaving a pixel pattern, plus any error or noiseintroduced by the transmission. Then, this pixel pattern may be comparedto the transmission-side pixel pattern. In an embodiment, the SoC 226directly compares the pixel pattern in the feedback signature to thepre-determined pixel pattern. In other examples, the SoC 226 comparesall, or a significant portion of, the feedback signature, which containsthe received image, to the modified image. In an embodiment, the SoC 226compares both the pixel patterns and the timestamps of the images. Thetime lag for the image transmission and the asynchronous nature of theclock for the SoC 226 and the clock for the display circuit 222 may betaken into account in comparing the transmission timestamp and thefeedback timestamp. The signature comparator 232 may subtract apre-defined time lag from the feedback timestamp to determine acompensated timestamp. Then, the compensated timestamp is compared tothe transmission timestamp. In one embodiment, the transmission-sidesignature and the feedback signature must match exactly to determine amatch. In another embodiment, the transmission-side signature and thefeedback signature are determined to be a match when they are similar,and to not be a match when they are not similar. For example, a sum ofabsolute difference may be determined between the transmitted pixelpattern and the pixel pattern, and a match is determined when the sum ofabsolute difference is below a predetermined threshold. In anembodiment, a match is found when the compensated timestamp is less thana first predetermined threshold of the transmission timestamp and thesum of absolute difference is less than a second predeterminedthreshold. The signature comparator 232 may send the results to the DSS110 to control further image transmission. Also, the signaturecomparator 232 may send the results for the CPU 106 and/or the DSP 108for analysis.

For example, the DSS 110 may halt image transmission when it detects afault, and continue image transmission when it does not detect a fault.In one embodiment, additionally or alternatively, the DSS 110 transmitsan error message to the display circuit 222, indicating the nature ormagnitude of the mismatch. In one embodiment, the pseudo-random addressgenerator 230, the pixel pattern generator 228, and the signaturecomparator 232 are implemented on a separate processor, or on the DSP108 or on the CPU 106. In other embodiments, they are implemented inspecialized hardware.

Histogram features used as a signature are resistant to minor noise.However, computing a histogram is complex and time consuming, and a CPUmay be poorly suited for computing a histogram. In an embodiment, ahardware circuit generates histograms on-the-fly. FIG. 9 illustrates ahistogram generation circuit 240, a digital circuit for computinghistograms. In an embodiment, the histogram generation circuit 240implements all or part of the signature generator 132 and the featuregenerator 176. The histogram generation circuit 240 may be used forother applications involving histogram computation. The histogramgeneration circuit 240 includes a register block 246, a register block252, an M way comparator 250, and a histogram merging circuit 256. Thehistogram generation circuit 240 receives an image 244 for histogramcomputation. The histogram generation circuit 240 updates multiplehistograms for a block of pixels in parallel, with one histogram perpixel. Then, the histogram generation circuit 240 combines the multipleparallel histograms, to generate one histogram for the block.

The register block 246 contains configuration registers 248, which storeranges for the blocks of the image 244 for histogram generation. In anexample, the configuration registers 248 contain the minimum x value,the maximum x value, the minimum y value, and the maximum y value, for ablock range, for example for block 242 of the image 244. Theconfiguration registers 248 output the values for the block 242 to the Mway comparator 250.

The register block 252 contains configuration registers 254, B−1 binthresholds. B, which is configurable, indicates the number of bins forthe histogram. The histogram merging circuit 256 contains histograms 262to 266 and a master histogram 270. The M way comparator 250 receives theblock 242 containing a block of M pixels, and the B−1 bin thresholdsfrom the configuration register 254. The M way comparator compares eachof the M pixels in parallel to the B−1 bin thresholds. The M waycomparator 250 produces M histograms, one histogram for each of M pixelsof the image, generating histograms 262 to 266 in the histogram mergingcircuit 256. Each of histograms 262 to 266 contains registers 264, oneregister for each of B bins, where the bins of each histogram indicatethe histogram values for one pixel of the M pixels. In an embodiment,each of histograms 262 to 266 has one bin with a value of 1, indicatingthe bin to which the corresponding pixel belongs, and the remaining binshave a value of 0. The histogram merging circuit 256 generates themaster histogram 270, with the bins 272, N bins, by adding the bins ofthe histograms 262 to 266 together. The histogram merging circuit 256sums the M histograms 262 to 266 to generate the master histogram 270. Mis an integer, for example 64, 256, or 1024. The value of N, for thebins 272, is configurable. The value of N may be configured by theregister block 252 to generate a 10-way histogram for one frame, and a64-way histogram for the next frame. This flexibility enables thehistogram generation circuit 240 to be immune to the effects ofcomputational complexity. The histogram generation circuit 240 is ableto compute sophisticated histograms with the same computational cost asstraightforward histograms. Also, the histogram updating occurs inparallel for M pixels, because of the M way comparator 250. Thehistogram data structure may be updated in parallel for M pixels, so thetime for total histogram is 1/M.

FIG. 10 illustrates a histogram comparison circuit 280 for computing andcomparing histograms for multiple images. The histogram comparisoncircuit 280 receives a histogram 282, for example for the output of theDSS. The histogram 282 contains bins 284, which are B bins. In oneembodiment, a histogram generation circuit, such as the histogramgeneration circuit 240, generates the histogram 282. The histogramcomparison circuit 280 also receives a histogram 286, for another image,for example for a feedback image for automotive display validation. Thehistogram 286 contains bins 288, which is also B bins. In oneembodiment, a histogram generation circuit, such as the histogramgeneration circuit 240, generates the histogram 286. In an example, thesame histogram generation circuit generates the histogram 282 and thehistogram 286. In another example, separate histogram generationcircuits generate the histogram 282 and the histogram 286.

The histogram 282 and the histogram 286 are fed to a differentiationcircuit 290, for example a sum of absolute difference circuit. Thedifferentiation circuit 290 compares the histogram 282 and the histogram286, to determine the sum of absolute difference between the histogram282 and the histogram 286, which indicates noise, for example from imagetransmission. The differentiation circuit 290 computes the sum ofabsolute difference between the histogram 282 and the histogram 286 bytaking the absolute value of the difference between the values of thebins of the histograms and summing these absolute value differences.When the sum of absolute difference is large, this may indicate asignificant degradation in the quality of the transmitted image.

A block 294 determines whether the sum of absolute difference fromdifferential circuit 290 is less than a threshold 292. The threshold 292may be a configurable threshold, which enables the configuration of avariable threshold, which is used to differentiate an acceptable levelof noise from a level of noise that indicates a transmission problem ora loss of accuracy. When the difference is less than the threshold 292,the histogram comparison circuit 280 proceeds to block 296, anddetermines that there is no error. On the other hand, when thedifference is greater than or equal to the threshold 292, the histogramcomparison circuit 280 proceeds to block 298, and determines that thereis an error.

Although example embodiments have been described in detail, it should beunderstood that various changes, substitutions and alterations can bemade thereto without departing from the spirit and scope of theinvention as defined by the appended claims.

What is claimed is:
 1. A method comprising: receiving, by asystem-on-a-chip (SoC) from a camera mounted on a vehicle, a first frameof a first image, wherein the SoC includes a first signature generatorand a signature comparator; generating, by the first signaturegenerator, a first signature using the first frame; transmitting, by theSoC to a display circuit over an interface cable, the first frame,wherein the display circuit includes a second signature generator, andwherein the display circuit is physically separate from the SoC;receiving, by the display circuit, a received first frame; generating,by the second signature generator, a second signature using the receivedfirst frame; transmitting, by the display circuit, the second signatureto the SoC; receiving, by the SoC from the display circuit, the secondsignature; determining, by the SoC, a comparison using the signaturecomparator; detecting, by the SoC, an error based on the comparison ofthe first signature not matching the second signature; and transmitting,by the SoC to the display circuit, a second frame of the first image, inresponse to the comparison determining that the first signature matchesthe second signature.
 2. The method of claim 1, wherein the firstsignature and the second signature are a cyclic redundancy check (CRC)value, a frame difference signature, or a secure hash algorithm (SHA)value.
 3. The method of claim 1, wherein the first signature is a firsttimestamp generated based on a first clock of the SoC, and wherein thesecond signature is a second timestamp generated base on a second clockof the display circuit.
 4. The method of claim 1, wherein the secondsignature is based on a third frame, and wherein the comparisondetermines whether the first frame matches the third frame.
 5. Themethod of claim 4, further comprising: selecting a pseudo-random blockof the first frame; and applying a pixel pattern to the pseudo-randomblock of the first frame, before transmitting the first frame; andwherein determining the comparison is based on whether the pseudo-randomblock of the first frame matches the pseudo-random block of the thirdframe.
 6. The method of claim 4, wherein determining whether the firstframe matches the third frame comprises: computing a first histogram ofthe first frame; computing a second histogram of the third frame; anddetermining whether the first histogram of the first frame matches thesecond histogram of the third frame.
 7. The method of claim 6, whereincomputing the first histogram is performed by a histogram generatorcircuit of the SoC, and wherein determining whether the first histogramof the first frame matches the second histogram of the third frame isperformed by a histogram comparison circuit of the SoC.
 8. The method ofclaim 4, wherein determining whether the first frame matches the thirdframe comprises: determining a first corner feature of the first frame;determining a second corner feature of the third frame; and determiningwhether the first corner feature of the first frame matches the secondcorner feature of the third frame.
 9. The method of claim 4, furthercomprising: scaling the first frame to match a scale of the third frame,before determining whether the first frame matches the third frame. 10.The method of claim 1, further comprising: receiving, by an interface ofthe display circuit, the first frame; storing, by a frame random accessmemory (RAM) of the display circuit, the first frame, as a stored image;displaying, on a display, the stored image; and capturing, by amonitoring camera, a view of the display, wherein the second signatureis the view of the display.
 11. The method of claim 1, furthercomprising: receiving, by an interface of the display circuit, the firstframe; storing, by a frame random access memory (RAM) of the displaycircuit, the first frame, as a stored image; displaying, on a display,the stored image, as a displayed image; and detecting, by a lightsensor, an intensity of a portion of the displayed image, wherein thesecond signature is a detected light intensity.
 12. The method of claim1, wherein the interface cable is a low voltage differential signaling(LVDS) cable, wherein the method further comprises: extracting, by anLVDS demultiplexer, an extracted image from the LVDS cable; andtransmitting, by the LVDS demultiplexer to the SoC, the extracted image,wherein the second signature is the extracted image.
 13. A circuitcomprising: an image signal processor configured to: receive a firstsignal from a camera; and produce a first image in response to the firstsignal from the camera; a display subsystem coupled to the image signalprocessor and configured to provide the first image to a displaycircuit, wherein the display circuit is configured to generate afeedback signature using a received first image, and wherein the displaycircuit is physically separate from the circuit; a processor coupled tothe image signal processor and configured to generate atransmission-side signature using the first image; and a signaturecomparator coupled to the processor and configured to: receive, from thedisplay circuit, the feedback signature; determine whether the feedbacksignature matches the transmission-side signature; when the feedbacksignature does not match the transmission-side signature, detect anerror in transmission of the first image; and when the feedbacksignature matches the transmission-side signature, cause the circuit totransmit, to the display circuit, a second image that is subsequent tothe first image.
 14. The circuit of claim 13, wherein each of thetransmission-side signature and the feedback signature includes at leastone value from a group consisting of: a cyclic redundancy check value, aframe difference value, and a secure hash algorithm value.
 15. Thecircuit of claim 13, wherein the determination of whether the feedbacksignature matches the transmission-side signature includes determiningwhether a difference between the feedback signature and thetransmission-side signature is greater than a threshold.
 16. The circuitof claim 13, wherein: the feedback signature includes a third imagebased on the first image; and the determination of whether the feedbacksignature matches the transmission-side signature includes determining adifference between the first image and the third image.
 17. A displaysystem comprising: a display circuit, the display circuit including: aninterface configured to receive, over an interface cable, a transmissionof a video image from a display subsystem as an image; a displaycontroller configured to display the image on a display device; and afeedback signature generator configured to: generate a feedbacksignature using the image; and transmit the feedback signature; and asystem-on-a-chip (SoC) physically separate from the display circuit, theSoC including: a processor configured to receive a plurality of videoframes from a camera; the display subsystem configured to process theplurality of video frames into the video image and to transmit the videoimage over the interface cable to the display circuit; a transmissionsignature generator configured to generate a transmission signatureusing the video image prior to the video image being transmitted to thedisplay circuit; and a signature comparator configured to receive thefeedback signature and the transmission signature, wherein the signaturecomparator is configured to determine whether the feedback signaturematches the transmission signature.
 18. The display system of claim 17further comprising: a frame random access memory (RAM) coupled to theinterface, the frame RAM configured to store the image; and lightemitting diode (LED) drivers coupled to the frame RAM, the LED driversconfigured to display the image on the display device.
 19. The displaysystem of claim 17, wherein: the feedback signature matches thetransmission signature when a difference between the transmissionsignature and the feedback signature is less than a predefinedthreshold.